OPENVPX COOLING SOLUTIONS An Elma White Paper 1 Prior to OpenVPX, many C4ISR (Command, Control, Communications, Computer, Intelligence, Surveillance and Reconnaissance) systems were built on parallel bus structures such as cPCI, VME, and largely custom imple- mentations. Specific systems were available at the box level, but did not support a common infrastructure or form factor across multiple boxes. In response to industry demands for standardization, a high-speed serial architecture - VPX (VITA 46 and VITA 65) was introduced, which brought new opportunities to mil/aero designs. VPX use of high-speed MultiGig backplane connectors vastly increased potential system bandwidth. This enabled increased high-speed serial I/O (for digital video, mass storage and FPGA interconnects), and switched serial fabrics with perfor- mance up to 10 Gbps, as well as rugged form factors with greater design flexibility. VPX enabled significant improvements in system speeds, reliability, upgradability, packaging and SWaP (Size, Weight and Power) for critical military applica- tions. It also provided greater bus structure common- ality across multiple boxes and even across the services. All this added performance contributes to thermal concerns however. In addition, OpenVPX introduces optical and RF signals to the backplane, removing these otherwise discrete connectors from the front of the cards. While the new backplane connections eliminate what would otherwise be a jumble of cables, the aggregate high-speed signals now traversing the backplane, rapidly heat up the system, exacerbating the already difficult to manage increasing temperatures. A recent aerospace application for example requires many RF inputs – 36 payload slots each with 16 RF signals and many large radar arrays will also require vast amounts of RF I/O signals. Some of the most complex cards are being used in signals intelligence applications for communica- tions and recording signals on the battlefield – including enemy communications – taking in audio inputs and triangulating the source of enemy fire. These high-perfor- mance applications require processor and FPGA system bandwidth that drive up the thermal load inside the chassis and demand new thermal management strategies. As higher performance systems are imple- mented, making the choice between 3U VPX and 6U VPX becomes a matter of what functionality can be packaged on the smaller card vs. the larger. As processors and FPGAs enable more capability, the 3U VPX form factor is favored for its reduced size and weight. That concentration of power in a smaller board complicates the choice to implement systems using a 3U card. Power density and the ability to dissipate heat becomes an issue. Purely convection or conduction solutions are pushed to their limits with existing cooling techniques defined by the standard. While OpenVPX offers significant improve- ments in field-deployed system signal integrity, speed and capability, it has created new challenges in space-constrained installations. Embedded sub-systems must sometimes be packaged to fit existing tight spaces in aircraft, ground vehicles, submarines, spacecraft, etc. As these systems have been implemented, the resulting increased heat has heavily impacted chassis and backplane designs, which in turn calls for new cooling options. 2 While conduction cooling as defined by VITA 48.2 and traditional convection cooling methods are still adequate for most current applications, the added complexity and heat generation of new boards and connectors, quickly push current system cooling methods to the limit. Recognizing this fast-developing issue, the VITA standards committees worked to define additional cooling methods under VITA 48. Among those, VITA 48.8 shows encouraging signs of becoming the most effective option. As VPX has grown in popularity, a number of VITA-ratified cooling standards have been developed: ĥ 48.4, liquid flow-through, probably the most efficient up to 350 watts per card ĥ 48.5, air flow-through, which has the advantage of being able to meter the air to specific cards ĥ 48.7, air flow-by ĥ 48.8, air flow-through cooling without sealing for small form-factor 3U and 6U VPX modules, ratified by ANSI (American National Standards Institute) in October 2017 The choice is based on consideration of the most practical design which may involve both the housing, the card heat sink and the chassis itself. The environment in which the boards will be developed and tested is commonly different than the final deployed unit so a lab chassis for example can typically rely on just fan cooling whereas a deployed unit might be required to use conduction cooling. Processor and memory chips tend to generate the most heat, with new processors drawing up to 200 watts at maximum functionality. Some cards, especially FPGAs, may approach 300 watts per slot, which can be a challenge even for liquid cooling. The Rise of VPX Cooling Requirements 3 With VITA 48.2 conduction cooling, power dissi- pation per slot is limited by the cold plate on which the chassis is mounted. Heat is conducted away from the hot components on the card by a heat spreader then transferred out to aluminum wedge locks attached to the edges of the card. The wedge locks are seated in aluminum card slots which pass heat to the chassis walls and finally to a cold plate. All conductive plates and surfaces are typically aluminum. The mechanical design of VITA 48.2 restricts air flow to the card edges where it’s cooling impact is limited and so conduction cooling and air cooling solutions are usually considered mutually exclusive. Both air and conduction methods could be used with liquid flow-through, a method increasingly considered for new designs. Liquid cooling however in any form requires significant infrastructure to implement, and so SWaP concerns rule out its use in some applications. Still, many payload boards require considerable power for their size, taking them out of conventional design strategies using conduction cooling, which is unable to dissipate the heat generated and convection cooling which starts to falter at higher altitudes where the air is thinner. VITA 48.8 uses air-flow-through cooling and its mechanical design supports air inlets at both card edges while routing air flow across the entire top surface of the boards. Conduction cooling methods in general provide a better system cooling alter- native to the complexity and infrastructure required by liquid cooling. Optifan 38 1.3-Heat Sink 1000Pa 38 2-Heat Sink 30mm Fin height 4 In the case of next-generation OpenVPX systems, the “C” in SWaP-C could be interpreted as “Cooling” as well as “Cost”. SWaP-C solutions are evolving as quickly as new OpenVPX applications are developed. This is especially true for rugged mil/aero systems. With higher speeds and compact designs, customer applications need to dissipate 50-to-75 percent more heat than before in roughly the same amount of space. Heat output is largely determined by the use of FPGA (Field-Programmable Gate Array) payloads in HPEC (High-Performance Embedded Computing) systems, especially in software-defined radios and radar systems. This adds to the increasing challenges of maintaining SWaP-C during the design process. This has intensified the emergence of VITA 48.8 as the cooling strategy of choice in the design of next-gener- ation OpenVPX systems. What the Future Holds for Chassis/Backplane Designs & Cooling Requirements Of value to military applications, especially in SWaP-C sensitive helicopters and unmanned aerial vehicles (UAVs), is the improved SWaP-C characteristics of VITA 48.8. Traditional card retainers and ejector/injector handles are replaced by lightweight jack screws, and since VITA 48.8 improved air flow design relies less on module-to-chassis conduction cooling, it allows the use of lighter composite chassis. Designers also now can incorporate fixed slot pitches of 1.0”, 1.2” and 1.5”, no longer being limited to VITA 48.5’s 1.52” maximum. That allows the use of alternate air flow arrange- ments, adding an air inlet at the card edge in addition to the conventional top edge inlet. Typically, a system architect within a company works with embedded card suppliers to address the functional requirements of the target design then approaches a packaging company such as Elma for a review of the power requirements and to propose the best cooling method. Because the VPX archi- tecture is so complex, the margin for error can be higher in a first-time implementation. The standard’s documentation has so metic- ulously defined the architecture however, that a company can define their sub-system’s requirements and outsource it. 5 Elma’s Role in Meeting Customer Needs New hardware convergence initiatives within the VPX community driven by the Department of Defense, enable greater compute density, which in turn is driving the need for advanced cooling methods. While VITA 48.8 is still a new tool for design engineers, it is expected to grow rapidly in application for next-gen- eration boards and backplanes. Supporting those initiatives, Elma’s 3U OpenVPX CMOSS backplane and development chassis provides the foundation to create systems optimized for performance, reduced SWaP and lower lifecycle costs with rapid technology insertion. The backplane includes precision radial network timing, plus slot profiles for SBCs, switches, radial clock(s) and expansion. Elma expects follow-on systems that will use derivative CMOSS architecture requiring high-power implementations. That will drive the use of alternate cooling techniques, such as VITA 48.8. Follow-on standards such as SOSA (Sensor Open Systems Architecture) also are now considering the need for cooling schemes beyond the standard VITA 48.X-based conduction cooled standards. A few companies began producing a limited number of VITA 48.8 boards in late 2017, and that number is expected to double during 2018. As key technical contributors to the OpenVPX development process from the beginning, Elma’s engineers are implementing new air-flow-through platform designs supporting the new VITA 48.8 boards, for those designing and producing high performance OpenVPX based embedded systems. This pertains to fully integrated chassis-level systems for both application development and deployment with chassis types ranging from 19” rackmount/desktop platforms to tower systems to Air Transport Racks (ATRs) 6 OpenVPX New Future with VITA 48.8 OpenVPX has allowed new definitions for VPX backplanes and systems, giving system architects and end users a far wider range of choices for use in critical high-speed applications. This helps to pave the way for more open architecture and multi-vendor interoperability in the future. It brings a set of standards that foster growth with technology over time, without requiring changes to system architecture. It uses adaptations within the standards themselves to enable new capabilities and build HPEC hardware. Finally, OpenVPX enables extraordinary leaps in aggregate system bandwidth and processing speeds which in turn drives the need for addressing the thermal challenges that result. VITA 48 and its underlying cooling standards, show promise in guiding the designs of advanced thermal management methods that will remove the roadblock now facing these high-speed systems. VITA 48.8, and its air flow-through cooling for small form-factor 3U and 6U VPX modules, shows particular promise and is being implemented by embedded system suppliers. Elma Electronic, with its field proven history of integrated systems and packaging for the embedded computing industry and leadership role in these emerging VITA standards, is well positioned to provide optimal solutions going forward. True interoperability in integrated systems design means building systems that accommodate board products from a variety of suppliers, while truly open design flexibility leads to an integrated solution better able to meet every aspect of the customer’s application requirements. As a single point of contact, Elma designs, builds, documents, supports and warrants the entire integrated system as a unit, including all components and sub-assemblies. Combined with its long history with VPX and OpenVPX – especially developing and implementing cooling solutions as each new component evolves – makes Elma a one-stop shop for 21st century mil/aero embedded systems and components. 7 About Elma Elma Electronic Inc. is a global manufacturer of commercial, industrial and rugged electronic products for embedded systems and application-ready platforms – from components, embedded boards, backplanes, chassis and enclosures, power supplies, to fully integrated subsystems. With one of the widest product ranges available in the embedded industry, Elma also offers standard and custom cabinets and enclosures as well as precision components such as rotary switches/encoders, LEDs, front panels and small cases. Elma leverages proven technology based on VITA, PICMG, and other standards-based architectures (i.e. OpenVPX, VME, CompactPCI Serial, ATCA, MicroTCA, COM Express and PCIe/104). Elma is also actively engaged in designing solutions for applications requiring smaller footprints. Elma Electronic manages entire projects from initial system architecture to specification, design, manufacturing and test through its worldwide production facilities and sales offices. The company serves the mil/aero, indus- trial, research, telecom, medical and commercial markets and is certified to ISO 9001 and AS 9100.